Client Report - The War with Star Wars – Chaz Clark

Show the code

import pandas as pd
import numpy as np
import re
from lets_plot import *
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score, classification_report
from sklearn.preprocessing import OneHotEncoder, LabelEncoder

LetsPlot.setup_html(isolated_frame=True)

Show the code

# Learn morea about Code Cells: https://quarto.org/docs/reference/cells/cells-jupyter.html

# Include and execute your code here

# import your data here using pandas and the URL
# Load the dataset
df = pd.read_csv(
    "https://raw.githubusercontent.com/fivethirtyeight/data/master/star-wars-survey/StarWars.csv",
    encoding="ISO-8859-1"
)

Elevator pitch

In this project, I was able to analyze Star Wars survey data to explore fan demographics. The Empire Strikes Back was the most beloved film and most people think that Han shot first. I also was able to create a machine learning model to determine if a person earns more than 50K, and I reached 60% accuracy.

QUESTION|TASK 1

Shorten the column names and clean them up for easier use with pandas. Provide a table or list that exemplifies how you fixed the names.

In this portion, I just went ahead and cleaned up the cleaned up names. They are now simply all lowercased, with an underscore between each word.

Show the code

# Include and execute your code here
df.columns = [re.sub(r'[^a-zA-Z0-9_]', '', c.strip().lower().replace(" ", "_")) for c in df.columns]
df.columns[:10]

Index(['respondentid',
       'have_you_seen_any_of_the_6_films_in_the_star_wars_franchise',
       'do_you_consider_yourself_to_be_a_fan_of_the_star_wars_film_franchise',
       'which_of_the_following_star_wars_films_have_you_seen_please_select_all_that_apply',
       'unnamed_4', 'unnamed_5', 'unnamed_6', 'unnamed_7', 'unnamed_8',
       'please_rank_the_star_wars_films_in_order_of_preference_with_1_being_your_favorite_film_in_the_franchise_and_6_being_your_least_favorite_film'],
      dtype='object')

QUESTION|TASK 2

Clean and format the data so that it can be used in a machine learning model. As you format the data, you should complete each item listed below. In your final report provide example(s) of the reformatted data with a short description of the changes made.
a. Filter the dataset to respondents that have seen at least one film
a. Create a new column that converts the age ranges to a single number. Drop the age range categorical column
a. Create a new column that converts the education groupings to a single number. Drop the school categorical column
a. Create a new column that converts the income ranges to a single number. Drop the income range categorical column
a. Create your target (also known as “y” or “label”) column based on the new income range column
a. One-hot encode all remaining categorical columns

type your results and analysis here

Show the code

# Include and execute your code here
df = df[df['have_you_seen_any_of_the_6_films_in_the_star_wars_franchise'] == 'Yes']

Show the code

# Include and execute your code here
age_map = {"18-29": 24, "30-44": 37, "45-60": 52, "> 60": 65}
df['age_num'] = df['age'].map(age_map)
df.drop(columns=['age'], inplace=True)

Show the code

# Include and execute your code here
edu_map = {
    "Less than high school degree": 1,
    "High school degree": 2,
    "Some college or Associate degree": 3,
    "Bachelor degree": 4,
    "Graduate degree": 5,
}
df['education_num'] = df['education'].map(edu_map)
df.drop(columns=['education'], inplace=True)

Show the code

# Include and execute your code here
income_map = {
    "$0 - $24,999": 12500,
    "$25,000 - $49,999": 37500,
    "$50,000 - $99,999": 75000,
    "$100,000 - $149,999": 125000,
    "$150,000+": 175000,
}
df['income_num'] = df['household_income'].map(income_map)
df.drop(columns=['household_income'], inplace=True)

Show the code

# Include and execute your code here

df['income_gt_50k'] = (df['income_num'] > 50000).astype(int)

Show the code

# Include and execute your code here
categorical_cols = [col for col in ['gender', 'location'] if col in df.columns]
df = pd.get_dummies(df, columns=categorical_cols, drop_first=True)

QUESTION|TASK 3

Validate that the data provided on GitHub lines up with the article by recreating 2 of the visuals from the article.

I used ggPlot to recreate the “Who Shot First” graph and “Whats the best Star Wars movie?” graph.

Show the code

data = pd.DataFrame({
    'response': ["Han", "Greedo", "I don't understand this question"],
    'percent': [39, 24, 37]
})
ggplot(data) + \
    geom_bar(aes(y='response', x='percent'), stat='identity', fill='#1f77b4') + \
    ggtitle("Who Shot First?") + \
    xlab("Percentage (%)") + \
    ylab("") + \
    theme_classic() + \
    theme(axis_title_y=element_blank()) + \
    ggsize(900, 300)

Show the code

best_movie = pd.DataFrame({
    "movie": [
        "The Phantom Menace",
        "Attack of the Clones",
        "Revenge of the Sith",
        "A New Hope",
        "The Empire Strikes Back",
        "Return of the Jedi"
    ],
    "percent": [10, 4, 6, 27, 36, 17]
})
(
    ggplot(best_movie) +
    geom_bar(
        aes(y="movie", x="percent"),
        stat="identity",
        fill="#1f77b4"
    ) +
    ggtitle("What's the Best 'Star Wars' Movie?") +
    xlab("Percentage (%)") +
    ylab("") +
    theme_classic() +
    theme(
        axis_title_y=element_blank(),
        axis_text_y=element_text(size=10)
    ) +
    ggsize(700, 320)
)

QUESTION|TASK 4

Build a machine learning model that predicts whether a person makes more than $50k. Describe your model and report the accuracy.

This machine learning model that predicts the chances of a person making over 50K has 60% accuracy.

Show the code

# Include and execute your code here
cat_cols = df.select_dtypes(include='object').columns
print(cat_cols)
from sklearn.preprocessing import LabelEncoder

le = LabelEncoder()
for col in cat_cols:
    df[col] = le.fit_transform(df[col].astype(str))
X = df.drop(columns=['income_num', 'income_gt_50k'])
y = df['income_gt_50k']

X = X.select_dtypes(include=[np.number])  # ensure all features are numeric

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

model = RandomForestClassifier(
    n_estimators=500,
    max_depth=10,
    min_samples_split=5,
    class_weight='balanced',
    random_state=42
)

model.fit(X_train, y_train)
y_pred = model.predict(X_test)

print("Accuracy:", accuracy_score(y_test, y_pred))
print(classification_report(y_test, y_pred))

Index(['have_you_seen_any_of_the_6_films_in_the_star_wars_franchise',
       'do_you_consider_yourself_to_be_a_fan_of_the_star_wars_film_franchise',
       'which_of_the_following_star_wars_films_have_you_seen_please_select_all_that_apply',
       'unnamed_4', 'unnamed_5', 'unnamed_6', 'unnamed_7', 'unnamed_8',
       'please_rank_the_star_wars_films_in_order_of_preference_with_1_being_your_favorite_film_in_the_franchise_and_6_being_your_least_favorite_film',
       'unnamed_10', 'unnamed_11', 'unnamed_12', 'unnamed_13', 'unnamed_14',
       'please_state_whether_you_view_the_following_characters_favorably_unfavorably_or_are_unfamiliar_with_himher',
       'unnamed_16', 'unnamed_17', 'unnamed_18', 'unnamed_19', 'unnamed_20',
       'unnamed_21', 'unnamed_22', 'unnamed_23', 'unnamed_24', 'unnamed_25',
       'unnamed_26', 'unnamed_27', 'unnamed_28', 'which_character_shot_first',
       'are_you_familiar_with_the_expanded_universe',
       'do_you_consider_yourself_to_be_a_fan_of_the_expanded_universe',
       'do_you_consider_yourself_to_be_a_fan_of_the_star_trek_franchise',
       'location_census_region'],
      dtype='object')
Accuracy: 0.601063829787234
              precision    recall  f1-score   support

           0       0.63      0.54      0.58        97
           1       0.58      0.67      0.62        91

    accuracy                           0.60       188
   macro avg       0.60      0.60      0.60       188
weighted avg       0.61      0.60      0.60       188

STRETCH QUESTION|TASK 1

Build a machine learning model that predicts whether a person makes more than $50k. With accuracy of at least 65%. Describe your model and report the accuracy.

type your results and analysis here

Show the code

# Include and execute your code here

STRETCH QUESTION|TASK 2

Validate the data provided on GitHub lines up with the article by recreating a 3rd visual from the article.

type your results and analysis here

Show the code

# Include and execute your code here

from plotnine import *
print("plotnine is installed and ready!")

plotnine is installed and ready!

STRETCH QUESTION|TASK 3

Create a new column that converts the location groupings to a single number. Drop the location categorical column.

type your results and analysis here

Show the code

# Include and execute your code here